Empirical Social Science Research Seminar Series
15 May 2024
University of Mannheim
👋 Camille Landesvatter
📍 Research Associate at the MZES, University of Mannheim
🏫 Studied social sciences and sociology at the University of Stuttgart and Mannheim
🎓 Started my PhD in Sociology in 2020 under the supervision of Florian Keusch and Paul Bauer (TrustME project)
🏠 Moved to Berlin in 2021 and visited the WZB for a research stay
Finished my PhD last month! 🎉
Audio Survey Responses: What insights can we gain by leveraging spoken responses in surveys to understand respondents’ views more deeply?
Emotion Analysis: What insights can we gain through automated emotion analysis of audio responses, and how do emotions influence political trust dynamics?
Political Trust: How can we measure political trust? And how can we use Audio Data and Emotion Analysis to support this task?
Cognitive Processing Modes:
Explanations:
Gavras et al. 2022: Audio formats facilitate the answer process by enabling open narration, intuitive and spontaneous answers (≠ intentional and conscious text answers).
Revilla et al. 2020: Speaking requires less effort than typing and voice formats make survey answering easier and quicker.
Increasing number of methods to analyze audio data
Intersection of Computer Science, Computational Linguistics and Computational Social Science
Examples of Methods:
Automatic Speech Recognition (“speech-to-text”)
Natural Language Processing
Speaker Diarization / Speaker Identification
Environmental Sound Analysis
Speech Emotion Recognition
Speech Emotion Recognition (SER) is part of the field of Automated Emotion Recognition, further intersecting with disciplines such as Neurocomputing and Affective Computing
Why use Speech for Emotion Analysis? → Recognizing emotions from text is difficult because it is stripped of all paralinguistic and acoustic features
Evolution of Models from traditional Machine Learning models (e.g., logistic regression) to deep learning based methods, e.g. SpeechBrain (Ravanelli et al. 2021)
The concept of emotions is not always clearly distinguished from similar phenomena such as mood, affect, and feeling. (Gabriel et al. 2023, 39)
Sentiment: the valence of a feeling (e.g., positive versus negative)
Emotions: a more complex and multi-dimensional state of feeling further characterized by their intensity as well as their cognitive evaluations (e.g., other-person control for anger)
Affect = “an umbrella term that is used to refer to both emotions and moods” (Lee, Dirks, and Campagna 2023, 549)
Conventional notion of political trust where trust judgments are made upon the basis of risk calculations and rational choice-making processes
Recently challenged by the idea of an “affect-based” form of political trust (e.g., Theiss-Morse and Barton 2017)
“A decision to trust a government organization may […] not always be conscious and/or rational.” (Grimmelikhuijsen 2012, 57)
Are individual trust judgments in surveys driven by affective components?
What thought processes and associations come to respondents’ minds when prompted to discuss ‘politics’ and their level of trust?
| Political Trust Question | “How often can you trust the federal government in Washington to do what is right?” | closed-ended, 4 categories (Always; Most of the time; About half of the time; Never; Don’t Know |
| Probing Question |
“The previous question was: ‘How often can you trust the federal government in Washington to do what is right?’. Your answer was: ‘About half of the time’. In your own words, please explain why you selected this answer.” |
open-ended, audio request, SVoice tool (Höhne, Gavras and Qureshi 2021) |
Self-administered web survey, September 6 to October 27, 2023
U.S. non-probability sample; \(n\)=1,474 with 491 audio open answers
quota-based (U.S. Census Bureau 2015) with challenges in obtaining sufficient participants in the oldest age category (58+)
We see how people feel affects how much they report to have trust in politics.
But we find few respondents to use emotional language in their answers and there is no consistent effect of emotions on reported trust score.
Two follow-up questions:
Overall, audio data plays a crucial role for studying societies, as spoken language is one of humanity’s most important means of communication, expression and information exchange in various fields (e.g., public speeches and debates, political talk shows and podcasts, press conferences).
→ We need more applied research to gain more experiences with this type of data.
Gabriel, Maier, Masch, and Renner. 2023. Political Leaders, the Display of Emotions, and the Public: An Empirical Study on the Coverage and Effects of Emotions in German Politics. Nomos.
Gavras, Höhne, Blom, and Schoen. 2022. “Innovating the collection of open-ended answers: The linguistic and content characteristics of written and oral answers to political attitude questions.” Journal of the Royal Statistical Society. Series A, 185(3):872-890.
Grimmelikhuijsen, Stephan. 2012. “Linking Transparency, Knowledge and Citizen Trust in Government: An Experiment.” International Review of Administrative Sciences 78(1):50–73.
Höhne and Gavras. 2022. “Typing or Speaking? Comparing Text and Voice Answers to Open Questions on Sensitive Topics in Smartphone Surveys.” Available at SSRN: https://ssrn.com/abstract=4239015 or http://dx.doi.org/10.2139/ssrn.4239015.
Lee, Kurt, and Rachel L. Campagna. 2023. “At the Heart of Trust: Understanding the Integral Relationship Between Emotion and Trust.” Group & Organization Management 48(2):546–80.
Lodge, Milton, and Charles S. Taber. 2013. The Rationalizing Voter. Cambridge University Press.
Lütters, Friedrich-Freksa, and Egger. 2018.“Effects of Speech Assistance in Online Questionnaires.” Presented at the General Online Research Conference, Vol. 18.
Ravanelli, Parcollet, Plantinga, et al. 2021. “SpeechBrain: A General-Purpose Speech Toolkit.” arXiv.
Revilla, Couper, Bosch, and Asensio. 2020. “Testing the Use of Voice Input in a Smartphone Web Survey.” Social Science Computer Review 38(2):207–24.
Theiss-Morse and Dona-Gene Barton. 2017. “Emotion, Cognition, and Political Trust.” Pp. 160–75 in Handbook on Political Trust. Edward Elgar Publishing.